1. Field of the Invention
The present invention relates to an apparatus for supporting evaluation of usability related to an operation with respect to a device or instrument which receives a user input and presents at least visual information in accordance with a content of the operation. An example of such an instrument would be a device that includes a graphical user interface (GUI).
2. Description of the Related Art
Instruments which present visual information related to user operation in order to supply various information, such as an instrument state and predetermined calculation result, to a user have become common and widespread. Examples of such instruments commonly found in most offices include copying machines, computers, and air conditioners. For example, most modern copying machines include a panel for setting the number of copies to be made and various copying functions (double-sided copying and reduction/enlargement), and a user operates the instrument based on the supplied visual information. Here, the visual information presented is not limited to dynamic information displayed on a liquid crystal display (LCD) such as information indicating the number of copies requested, but includes printed and fixed information, such as, for example, numerals “1”, “2”, . . . printed on a ten key keypad disposed for setting the number of copies to be made in the copying machine.
As instrument functionality has grown more sophisticated, operations have increasingly become complicated. International efforts have been made to facilitate improvement of instruments by providing indexes for evaluating the usability of instruments, such as user-friendliness of using the instrument, effectiveness of use of the instrument, efficiency, and satisfaction given to a user of the instrument. For example, the ISO 9241-11 standard (Ergonomic requirements for office work with visual display terminals (VDTs)-Part 11: Guidance on usability) defines evaluation measures related to usability from the viewpoints of effectiveness, efficiency, and satisfaction, and device usability can be enhanced through such evaluation.
Here, usability indicates the degree of effectiveness, efficiency, and user satisfaction in achieving a designated object under designated circumstances of a certain product by a designated user. The effectiveness indicates accuracy and completeness in achieving a designated result, efficiency measures the amount of resources consumed in association with the accuracy and completeness in achieving the target by the user, and satisfaction measures comfort and acceptability for a user who uses a computer system and the computer system, that is, the degree to which a user is free from frustration and is able to maintain a positive attitude toward product use.
Known methods of performing evaluation of a task using information from users who have completed the relevant task include questionnaires in which users independently read and answer questions and interviews where users are asked questions and provide answers to evaluators. With a questionnaire, although a specific event at a task achieving time or an entire impression at a task end time can be better understand, a relationship between the operation content and problem is often unclear, and it is often difficult to use the information to improve the device.
There are known apparatuses for recording a user operation log and diagrammatically displaying the log record to facilitate analysis (e.g., Patent Document 1:Japanese Patent Application Laid-Open No. 8-153022). Moreover, there are techniques for recording and reproducing the operation log between the system which is an evaluation object, and the user and listing and displaying the state log of the system (e.g., Patent Document 2:Japanese Patent Application Laid-Open No. 2001-51876).
Furthermore, there is another related art system in which user testing is employed. In user testing, a test subject regarded as a user actually operates the instrument, a test subject which attempting to perform a task is observed by an observer, the observer presumes what the test subject is going to do, and the observer notes steps at which the usability drops.
For example, when it is observed that the test subject's eyeshot moves over an operation panel, it is presumed that the test subject is searching the panel for information or a means to input a desired command. When this takes an excessively long amount of time, it is presumed that the operation of the panel is not clear and the panel design is not efficient for operation, and therefore the usability of such a panel is low.
However, in these related-art methods of evaluating usability, an evaluator must refer to records of the operation log, state log of the system, and test subject psychological state to conduct the evaluation. As a result, the evaluator's arbitrary judgment is easily included in the evaluation and data, and judgments often depend on the evaluator's acquaintance.
When there are a plurality of evaluators, particularly when there are many evaluation objects, a unified evaluation is not easily obtained, and portions requiring improvement are not easily distinguished. Moreover, to obtain the questionnaire or interview information from the test subject, only the content that the test subject is aware of can be extracted. Therefore, there is a large possibility that a test subject will unconsciously edit their thoughts or comments in order to describe the psychological state after the end of the operation. Furthermore, in the evaluation method of the usability, the indexes of the usability in conformity with ISO9241-11 such as effectiveness, efficiency, and satisfaction are not always obtainable.
Additionally, even if data corresponding to the indexes such as effectiveness, efficiency, and satisfaction is obtained, the content or definition of the index differs with each type of business, and therefore it is further difficult to unify the evaluation. Moreover, effectiveness, efficiency, and satisfaction can be defined in each situation of use of the evaluation object. Therefore, when the effectiveness, efficiency, and satisfaction are not obtained according to the same definitions among the objects to be compared/evaluated, the meaning of quantification and evaluation is weakened. Additionally, there has been a demand that a specific component of a certain task be evaluated as a factor of deterioration of usability in order to improve that component. Therefore, evaluator's burden is heavy, and sufficient training with respect to the evaluator is necessary.
Additionally, there remains an unfulfilled demand for the ability to evaluate a task component which is adversely effecting the usability.